1. Field of the Invention
The present invention relates to methods of coding and decoding image sequences and to coding and decoding devices for those purposes.
2. Description of the Related Art
MPEG Standards of the ISO (MPEG-1, MPEG-2 and MPEG-4) or H.26× Standards of the ITU-T (H.261, H.263) are currently used for video coding. It is possible to compress a digital video sequence with a predetermined picture size and picture repetition rate and to reconstruct the original sequence from the resulting bit stream with quality loss. The quality loss depends essentially on the compression degree, i.e. the ratio of the input data rate to the output data rate of the decoder. However the image content to be coded and the coding process that is used have a great influence on the quality of the reconstructed video. The quality can be controlled in two ways in the above-described method according to the requirements of the application. Either an approximately constant image quality is established, in which coding with a variable bit rate results because of the fluctuating complexity of the image content, or an approximately constant data rate is established, whereby a fluctuating picture quality results. Most applications require a coding according to the second case, in which the data rate is controlled.
The respective encoder performs the data rate control. However none of the standards provide norms. However mechanisms are set up and made available for influencing the reconstruction quality and thus the resulting data rate. The most common control methods involve variation of the quantization parameter (QP) for coding the transformation coefficient. In MPEG-4 Standard as described in “MPEG-4 Video Transmission over DAB/DMB: Joined Optimization of Encoding and Format Conversion” by A.Smolić, K. Rümmler, J. -R. Ohm, R. Schäfer and S. Bauer, Proc. MOMUC' 98, International Workshop on Mobile Multimedia Communication, Berlin, Germany Oct. 12–14, 1998, a change of the parameter with reference to the previous quantization parameter (QP) can be transmitted with each macroblock. The absolute QP can be transmitted in MPEG-2 and H.263 according to ISO/IEC/JTC1/SC29/WG11, in “MPEG-4 Video VM 16.0”, Doc. N3312, Noordwijkerhout, Niederlands, March 2000. The quantization parameter QP can take 32 different values, whereby a macroblock with 32 different quality and bit rates can be coded. The change of the number of images to be coded is one additional element for rate control. A maximum bit rate at which the coding should proceed is established at the beginning of the coding by the user, for example only every second image of the image sequence should be coded. On the other hand, individual images can be omitted as required in some coding processes. These so-called frame skips are most used when an additional increase of the QP is no longer possible or leads to an unacceptable picture quality.
In video coding with a predetermined fixed bit rate it is necessary to control the encoder so that the required bit rate is maintained and at the same time the best image quality is achieved. This encoder control is called rate control. The primary means for adjusting the bit rate is the quantization (quantization parameter QP), which varies according to the already consumed bits. Coarse quantization of the structure and motion of complex video material is easy. This can generally lead to interfering or troublesome block artifacts with very complex content. In the extreme case however the required bit rate cannot be maintained in spite of the coarsest quantization. In this case individual images may be omitted, i.e. that is not coded and transmitted. Instead of this the previous image is frozen. This sort of frame skipping generally leads to extremely annoying bucking or jerking effects, especially with video in line-jump format, in which the motion appears jerky or discontinuous because of the temporal displacement of both half images.
EP 0 346 635 B1 discloses that it is known to transfer or transmit only that image block which has the greatest residual errors according to the filling state of the bit stream buffer during the coding.
In order to prevent frame skipping the possibility exists to code the image sequence in one of the foregoing formats, which practically exclude the occurrence of frame skipping, i.e. to use a suitably reduced temporal and spatial resolution. Experiments have generally shown that the subjective quality increases with increasing resolution of the coding format, as explained by A.Smolić, ibid. Thus an optimization problem exists for the coding format, whose opposing factors are the avoidance of frame skipping in standard video image sequences with both complex and coarse quality. The use of a fixed coding format must therefore involve a compromise.